Skip to content

Support arithmetic operations for dense_vectors: scalar version#141060

Open
kkharbas wants to merge 10 commits intoelastic:mainfrom
kkharbas:dense_vector-arithmetic-scalar
Open

Support arithmetic operations for dense_vectors: scalar version#141060
kkharbas wants to merge 10 commits intoelastic:mainfrom
kkharbas:dense_vector-arithmetic-scalar

Conversation

@kkharbas
Copy link
Contributor

@kkharbas kkharbas commented Jan 21, 2026

Adds support for arithmetic operations - '+', '-', '*' and '/' when one operand is a dense_vector and other is scalar.

Performs the arithmetic operation on each element of the dense_vector against the scalar operand. The scalar operand should be a constant value, scalar fields are not supported.
✔️ dense_vector + scalar constant
dense_vector + scalar field

Closes #140538

Copy link
Member

@carlosdelest carlosdelest left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hey @kkharbas ! I don't think we need to have a

dense_vector + field

so we operate on each dense_vector field over each scalar field on the other column, but:

dense_vector + scalar constant

so we use the same scalar constant to operate on every dense_vector field row.

So, we could do something like:

EVAL my_negated_vectors = my_vector_field * -1

or

EVAL my_added_vectors = my_vector_field + 3.0

Right now I don't see the use case for operating on a row by row basis for scalars - and I think in any case using a single scalar constant would be a more frequent use case, and simpler to implement for now.

We may come back to this implementation, but I think for now we should reduce the scope and look for single numeric constants to be applied.

What do you think?

@kkharbas
Copy link
Contributor Author

kkharbas commented Jan 22, 2026

Hey @kkharbas ! I don't think we need to have a

dense_vector + field

so we operate on each dense_vector field over each scalar field on the other column, but:

dense_vector + scalar constant

so we use the same scalar constant to operate on every dense_vector field row.

So, we could do something like:

EVAL my_negated_vectors = my_vector_field * -1

or

EVAL my_added_vectors = my_vector_field + 3.0

Right now I don't see the use case for operating on a row by row basis for scalars - and I think in any case using a single scalar constant would be a more frequent use case, and simpler to implement for now.

We may come back to this implementation, but I think for now we should reduce the scope and look for single numeric constants to be applied.

What do you think?

That makes total sense. I was just focussing on completeness of the operations but usability wise its fair to only support scalar constant.

@github-actions
Copy link
Contributor

github-actions bot commented Feb 9, 2026

ℹ️ Important: Docs version tagging

👋 Thanks for updating the docs! Just a friendly reminder that our docs are now cumulative. This means all 9.x versions are documented on the same page and published off of the main branch, instead of creating separate pages for each minor version.

We use applies_to tags to mark version-specific features and changes.

Expand for a quick overview

When to use applies_to tags:

✅ At the page level to indicate which products/deployments the content applies to (mandatory)
✅ When features change state (e.g. preview, ga) in a specific version
✅ When availability differs across deployments and environments

What NOT to do:

❌ Don't remove or replace information that applies to an older version
❌ Don't add new information that applies to a specific version without an applies_to tag
❌ Don't forget that applies_to tags can be used at the page, section, and inline level

🤔 Need help?

Adds support for arithmetic operations - '+', '-', '*' and '/' when one operands is a dense_vector and other is scalar. Performs the arithmetic operation
on each element of the dense_vector against the scalar operand.

Closes elastic#140538
@kkharbas kkharbas marked this pull request as ready for review February 10, 2026 06:55
@elasticsearchmachine elasticsearchmachine added the needs:triage Requires assignment of a team area label label Feb 10, 2026
Copy link
Member

@carlosdelest carlosdelest left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I have a question on the approach - instead of dealing separately with scalar operands in a separate class, convert them via an ExpressionEvaluator to dense vectors so we can operate on them with the current infra.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Having a specific class for handling operating with doubles looks like something we should try to avoid.

I wonder about creating a ToFloat internal function for dealing with this, similar to other conversion functions like ToDouble. This would allow us to use ToFloat as part of the evaluator chain that the factory creates when we find a numeric type that we want to convert to floats, and then deal with doing float to float operations.

WDYT?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I wonder if we can avoid having a specific evaluator for doubles, and adding a specific DenseVectorBinaryEvaluator.

I was thinking on having a ExpressionEvaluator that would convert a numeric block into a dense_vector. That way, we could wrap the numeric block using this converter before it is processed by the DenseVectorsEvaluator, and thus reducing the problem to an operation on two dense vectors which we already have solved.

WDYT?

@kkharbas
Copy link
Contributor Author

kkharbas commented Feb 10, 2026

I have a question on the approach - instead of dealing separately with scalar operands in a separate class, convert them via an ExpressionEvaluator to dense vectors so we can operate on them with the current infra.

I had given this approach some consideration but ran into hiccups.
One is we throw error when vector dimensions don't match. Would it be possible to distinguish between the case when an operand is a scalar converted to vector or a vector with dimension == 1?

I will take a fresh look again at this approach.

@kkharbas
Copy link
Contributor Author

I have a question on the approach - instead of dealing separately with scalar operands in a separate class, convert them via an ExpressionEvaluator to dense vectors so we can operate on them with the current infra.

I had given this approach some consideration but ran into hiccups. One is we throw error when vector dimensions don't match. Would it be possible to distinguish between the case when an operand is a scalar converted to vector or a vector with dimension == 1?

I will take a fresh look again at this approach.

I tried out this approach of using existing conversion evaluators, e.g. ToDenseVectorFromIntEvaluator, its a neat way to use existing dense_vector support.

Few thoughts and questions on this,

  1. Evaluation logic is slightly different for when both arguments are dense_vectors vs one is scalar, e.g. validating dimensions and getting N vs 1 values from the block. The execution path to take depends on the expression and can be injected into the evaluator as isLhsScalar and isRhsScalar boolean parameters. Question/concern is - is it a significant performance concern doing these checks during evaluation (once per page)?
  2. This approach can be applied to scalar fields too (not just constants) without adding much complexity. If the concern was just related to complexity, does it make sense to support non-literals?

@kkharbas kkharbas added Team:Search Relevance Meta label for the Search Relevance team in Elasticsearch :Search Relevance/ES|QL Search functionality in ES|QL labels Feb 18, 2026
@elasticsearchmachine elasticsearchmachine removed the needs:triage Requires assignment of a team area label label Feb 18, 2026
@elasticsearchmachine
Copy link
Collaborator

Pinging @elastic/es-search-relevance (Team:Search Relevance)

@kkharbas
Copy link
Contributor Author

test this please

@elasticsearchmachine
Copy link
Collaborator

Hi @kkharbas, I've created a changelog YAML for you.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

>enhancement :Search Relevance/ES|QL Search functionality in ES|QL Team:Search Relevance Meta label for the Search Relevance team in Elasticsearch v9.4.0

Projects

None yet

Development

Successfully merging this pull request may close these issues.

ES|QL - Support dense_vector arithmetic functions: scalar versions

3 participants

Comments